Weekly Exercises #5

library(tidyverse)     # for data cleaning and plotting
library(gardenR)       # for Lisa's garden data
library(lubridate)     # for date manipulation
library(openintro)     # for the abbr2state() function
library(palmerpenguins)# for Palmer penguin data
library(maps)          # for map data
library(ggmap)         # for mapping points on maps
library(gplots)        # for col2hex() function
library(RColorBrewer)  # for color palettes
library(sf)            # for working with spatial data
library(leaflet)       # for highly customizable mapping
library(ggthemes)      # for more themes (including theme_map())
library(plotly)        # for the ggplotly() - basic interactivity
library(gganimate)     # for adding animation layers to ggplots
library(transformr)    # for "tweening" (gganimate)
library(gifski)        # need the library for creating gifs but don't need to load each time
library(shiny)         # for creating interactive apps
library(ggimage)
library(scales)
theme_set(theme_minimal())

# SNCF Train data
small_trains <- read_csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2019/2019-02-26/small_trains.csv") 

# Lisa's garden data
data("garden_harvest")

# Lisa's Mallorca cycling data
mallorca_bike_day7 <- read_csv("https://www.dropbox.com/s/zc6jan4ltmjtvy0/mallorca_bike_day7.csv?dl=1") %>% 
  select(1:4, speed)

# Heather Lendway's Ironman 70.3 Pan Am championships Panama data
panama_swim <- read_csv("https://raw.githubusercontent.com/llendway/gps-data/master/data/panama_swim_20160131.csv")

panama_bike <- read_csv("https://raw.githubusercontent.com/llendway/gps-data/master/data/panama_bike_20160131.csv")

panama_run <- read_csv("https://raw.githubusercontent.com/llendway/gps-data/master/data/panama_run_20160131.csv")

#COVID-19 data from the New York Times
covid19 <- read_csv("https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-states.csv")

data_site <- 
  "https://www.macalester.edu/~dshuman1/data/112/2014-Q4-Trips-History-Data.rds" 
Trips <- readRDS(gzcon(url(data_site)))

Put your homework on GitHub!

Go here or to previous homework to remind yourself how to get set up.

Once your repository is created, you should always open your project rather than just opening an .Rmd file. You can do that by either clicking on the .Rproj file in your repository folder on your computer. Or, by going to the upper right hand corner in R Studio and clicking the arrow next to where it says Project: (None). You should see your project come up in that list if you’ve used it recently. You could also go to File –> Open Project and navigate to your .Rproj file.

Instructions

• Put your name at the top of the document.

• For ALL graphs, you should include appropriate labels.

• Feel free to change the default theme, which I currently have set to theme_minimal().

• Use good coding practice. Read the short sections on good code with pipes and ggplot2. This is part of your grade!

• NEW!! With animated graphs, add eval=FALSE to the code chunk that creates the animation and saves it using anim_save(). Add another code chunk to reread the gif back into the file. See the tutorial for help.

• When you are finished with ALL the exercises, uncomment the options at the top so your document looks nicer. Don’t do it before then, or else you might miss some important warnings and messages.

Warm-up exercises from tutorial

1. Choose 2 graphs you have created for ANY assignment in this class and add interactivity using the ggplotly() function.

beet_harvest_graph <- garden_harvest %>% 
  filter(vegetable == "beets") %>% 
  group_by(date, variety) %>% 
  summarize(wt_hvst_grams = sum(weight)) %>% 
  group_by(variety) %>% 
  mutate(wt_lbs = wt_hvst_grams * 0.00220462, 
         cum_wt_hvst_lbs = cumsum(wt_hvst_grams * 0.00220462)) %>% 
  ggplot(aes(x = date, y = cum_wt_hvst_lbs, color = variety)) +
  geom_line() + 
  scale_color_manual(values = c( "#D1BE1D", 'dark green', "#5E1B4C")) +
  labs(title = "Cumulative Harvest Weight (in lbs) of Beets by Variety", 
       x = "", 
       y = "", 
       color = "Beet Variety")

ggplotly(beet_harvest_graph)

bike_rental_DOW <- Trips %>% 
  mutate(weekday = wday(sdate, label = TRUE), 
         hour = hour(sdate), 
         minute = minute(sdate), 
         time = hour + (minute/60)) %>% 
  ggplot(aes(x = time)) +
  geom_density(aes(fill = client), alpha = .5, color = NA) +
  facet_wrap(vars(weekday)) +
  labs(title = "Distribution of Bike Rentals by Time of Day and Weekday",  
       x = "Time of Day (Military Time)", 
       y = "Density", 
       color = "", 
       fill = "Client Type")

ggplotly(bike_rental_DOW)

2. Use animation to tell an interesting story with the small_trains dataset that contains data from the SNCF (National Society of French Railways). These are Tidy Tuesday data! Read more about it here.

small_trains %>% 
  filter(departure_station %in% c("BORDEAUX ST JEAN","AVIGNON TGV", "POITIERS"), 
         year == "2017") %>% 
  ggplot(aes(x = month, y = total_num_trips, color = departure_station)) +
  geom_point() +
  xlim(1, 12) +
  labs(title = "Deparatures from the 3 Busiest Stations in France in 2017", 
       x = "Month", 
       y = "Number of Departures", 
       color = "Departure Station") +
  theme(legend.position = "top",
        legend.title = element_blank()) +
  transition_time(month) +
  shadow_wake(wake_length = .3)

#anim_save("BusyStationDepartures.gif")

knitr::include_graphics("BusyStationDepartures.gif")

Garden data

3. In this exercise, you will create a stacked area plot that reveals itself over time (see the geom_area() examples here). You will look at cumulative harvest of tomato varieties over time. You should do the following:

• From the garden_harvest data, filter the data to the tomatoes and find the daily harvest in pounds for each variety.
  
• Then, for each variety, find the cumulative harvest in pounds.
  
• Use the data you just made to create a static cumulative harvest area plot, with the areas filled with different colors for each vegetable and arranged (HINT: fct_reorder()) from most to least harvested (most on the bottom).
  
• Add animation to reveal the plot over date.

I have started the code for you below. The complete() function creates a row for all unique date/variety combinations. If a variety is not harvested on one of the harvest dates in the dataset, it is filled with a value of 0.

garden_harvest %>% 
  filter(vegetable == "tomatoes") %>% 
  group_by(date, variety) %>% 
  summarize(daily_harvest_lb = sum(weight)*0.00220462) %>% 
  ungroup() %>% 
  complete(variety, date, fill = list(daily_harvest_lb = 0)) %>% 
  group_by(variety) %>% 
  mutate(daily_cum_harvest_lb = cumsum(daily_harvest_lb)) %>% 
  ggplot(aes(x = date, y = daily_cum_harvest_lb, fill = fct_reorder(variety, daily_cum_harvest_lb, max))) +
  geom_area() + 
  labs(title = "Cumulative Daily Tomato Harvest", 
       x = "", 
       y = "Harvest (in lbs)", 
       fill = "Tomato Variety") + 
  theme(legend.position = "left") +
  transition_reveal(date)

#anim_save("daily_harvest.gif")

knitr::include_graphics("daily_harvest.gif")

Maps, animation, and movement!

4. Map my mallorca_bike_day7 bike ride using animation! Requirements:

• Plot on a map using ggmap.
  
• Show “current” location with a red point.
• Show path up until the current point.
  
• Color the path according to elevation.
  
• Show the time in the subtitle.
  
• CHALLENGE: use the ggimage package and geom_image to add a bike image instead of a red point. You can use this image. See here for an example.
• Add something of your own! And comment on if you prefer this to the static map and why or why not.

mallorca_map <- get_stamenmap(
    bbox = c(left = 2.28, bottom = 39.41, right = 3.03, top = 39.8), 
    maptype = "terrain",
    zoom = 11)

bike_icon <- "https://raw.githubusercontent.com/llendway/animation_and_interactivity/master/bike.png"

mallorca_update <- mallorca_bike_day7 %>% 
  mutate(icon = bike_icon)

ggmap(mallorca_map) +
  geom_path(data = mallorca_bike_day7, 
            aes(x = lon, y = lat, color = ele),
            size = .5) +
  scale_color_viridis_c(option = "magma") +
  theme_map() +
  geom_image(data = mallorca_update, 
             aes(x = lon, y = lat, image = icon), 
             size = .1) + 
  labs(title = "Bike Ride With Elevation", 
       color = "Elevation", 
       subtitle = "Date/Time: {frame_along}") +
  transition_reveal(time) +
  shadow_mark(past = TRUE)

#anim_save("mallorca_bike.gif")

knitr::include_graphics("mallorca_bike.gif")

Discussion: I like this animated, dynamic map more than the static original. The temporal component allows me to better understand the elevation change and get a sense of how fast you were going. I think it can be a fine line to walk, though, where the animated graphic also makes it difficult to really analyze the complete data as I’m more concerned with the animation itself. So, it depends on the purpose of the graph and what you’re hoping to convey.

5. In this exercise, you get to meet my sister, Heather! She is a proud Mac grad, currently works as a Data Scientist at 3M where she uses R everyday, and for a few years (while still holding a full-time job) she was a pro triathlete. You are going to map one of her races. The data from each discipline of the Ironman 70.3 Pan Am championships, Panama is in a separate file - panama_swim, panama_bike, and panama_run. Create a similar map to the one you created with my cycling data. You will need to make some small changes: 1. combine the files (HINT: bind_rows(), 2. make the leading dot a different color depending on the event (for an extra challenge, make it a different image using `geom_image()!), 3. CHALLENGE (optional): color by speed, which you will need to compute on your own from the data. You can read Heather’s race report here. She is also in the Macalester Athletics Hall of Fame and still has records at the pool.

complete_triathalon <- 
  bind_rows(panama_bike, panama_run, panama_swim)

panama_map <- get_stamenmap(
  bbox = c(left = -79.6278, bottom = 8.8891, right = -79.4407, top = 9.0151), 
    maptype = "terrain",
    zoom = 13)

ggmap(panama_map) +
  geom_path(data = complete_triathalon, 
            aes(x = lon, y = lat, color = event),
            size = .5) +
  geom_point(data = complete_triathalon, 
             aes(x = lon, y = lat, color=event),
             size = 2)+
  theme_map() +
  labs(title="Heather Lenway Ironman 70.3 Pan Am Championships",
    subtitle="Time:{frame_along}", 
    color = "Event")+
  transition_reveal(time)+
  shadow_mark(past=FALSE)

#anim_save("triathalon.gif")

knitr::include_graphics("triathalon.gif")

COVID-19 data

6. In this exercise, you are going to replicate many of the features in this visualization by Aitish Bhatia but include all US states. Requirements:

• Create a new variable that computes the number of new cases in the past week (HINT: use the lag() function you’ve used in a previous set of exercises). Replace missing values with 0’s using replace_na().
  
• Filter the data to omit rows where the cumulative case counts are less than 20.
  
• Create a static plot with cumulative cases on the x-axis and new cases in the past 7 days on the y-axis. Connect the points for each state over time. HINTS: use geom_path() and add a group aesthetic. Put the x and y axis on the log scale and make the tick labels look nice - scales::comma is one option. This plot will look pretty ugly as is.
• Animate the plot to reveal the pattern by date. Display the date as the subtitle. Add a leading point to each state’s line (geom_point()) and add the state name as a label (geom_text() - you should look at the check_overlap argument).
  
• Use the animate() function to have 200 frames in your animation and make it 30 seconds long.
• Comment on what you observe.

covid_anim <- covid19 %>% 
  group_by(state) %>%
  mutate(seven_day_lag = lag(cases, n = 7, oder_by = date), 
         seven_day_lag = replace_na(seven_day_lag, 0), 
         new_weekly_cases = cases - seven_day_lag) %>% 
  filter(cases > 20) %>% 
  ggplot(aes(x = cases, y = new_weekly_cases, group = state)) +
  geom_point(aes(x = cases, y = new_weekly_cases)) +
  geom_path(aes(group = state)) +
  geom_text(aes(label = state), check_overlap = TRUE) +
  scale_x_log10(labels = comma)+
  scale_y_log10(labels = comma) +
  labs(title = "Covid Cases Trajectory", 
       subtitle = "Date: {frame_along}", 
       x = "Total Cases", 
       y = "New Weekly Cases") +
  transition_reveal(date) +
  shadow_mark(past = TRUE) 

animate(covid_anim, nframes = 200, duration = 30)

#anim_save("covid_trajectory.gif")

knitr::include_graphics("covid_trajectory.gif")

Discussion: This animation is hard to read for a number of reasons. Visually, it’s very difficult to discern which state I’m looking at and pick apart any meaningful trends for any of the states since they overlap so much. If we could pare down the number of states, this would be a much more effective animation. The other reason that this graph is difficult to read is because its log scaled. This is actually a smart choice, but makes it very difficult to interpret correctly. This scaling doesn’t allow an average viewer to understand the absolute sheer rise in the number of cases. Nonetheless, we can see from this graph that there is a consistent rise in cases, with points of improvement. It’s difficult to see really granular trends, though I suspect if we were to zoom in on the last couple months and continued graphing this going forward, we’d notice trends of much slower growth. Perhaps the most striking part of this visualization is the quick rapid rise in cases we saw in the early months of the pandemic. State after state saw rapid increases and generally appear to be slowing their rates in later months.

7. In this exercise you will animate a map of the US, showing how cumulative COVID-19 cases per 10,000 residents has changed over time. This is similar to exercises 11 & 12 from the previous exercises, with the added animation! So, in the end, you should have something like the static map you made there, but animated over all the days. The code below gives the population estimates for each state and loads the states_map data. Here is a list of details you should include in the plot:

• Put date in the subtitle.
  
• Because there are so many dates, you are going to only do the animation for all Fridays. So, use wday() to create a day of week variable and filter to all the Fridays.
  
• Use the animate() function to make the animation 200 frames instead of the default 100 and to pause for 10 frames on the end frame.
  
• Use group = date in aes().
  
• Comment on what you see.

census_pop_est_2018 <- read_csv("https://www.dropbox.com/s/6txwv3b4ng7pepe/us_census_2018_state_pop_est.csv?dl=1") %>% 
  separate(state, into = c("dot","state"), extra = "merge") %>% 
  select(-dot) %>% 
  mutate(state = str_to_lower(state))

states_map <- map_data("state")

covid19_with_pop_est <- covid19 %>%
  mutate(state = tolower(state)) %>% 
  left_join(census_pop_est_2018, 
            by = c("state")) %>% 
  mutate(dow = wday(date, label = TRUE)) %>% 
  filter(dow == "Fri")
  

covid_standardized <- covid19_with_pop_est %>% 
  ggplot() +
  geom_map(map = states_map,
           aes(map_id = state,
               fill = (cases / est_pop_2018) * 10000, 
               group = date)) +
  expand_limits(x = states_map$long, y = states_map$lat) + 
  theme_map() + 
  labs(title = "COVID-19 Cases per 10,000 People by State",
       subtitle = "Date: {frame_time}",
       fill = "Total Cumulative Cases per 10000 People by State") +
  theme(legend.position = "bottom") +
  transition_time(date) 

animate(covid_standardized, nframes = 200, end_pause = 10)

#anim_save("covid_standardized.gif")

knitr::include_graphics("covid_standardized.gif")

Observations: We can see from this graph the steady rise in cases per 10,000 across the US. It’s interesting to see how the relative rates of cases per capita change as we move across the calendar. Only until later in the summer do we start seeing states hit the high end of the scale of the visualization. Perhaps what’s most interesting to me about this visualization, is the way you can focus on any given state and see how their cases per capita change over time. It’s quite powerful to see the colors change– it’s its own story within the story that captures the persistent takeover COVID has had throughout the US.

Your first shiny app (for next week!)

NOT DUE THIS WEEK! If any of you want to work ahead, this will be on next week’s exercises.

8. This app will also use the COVID data. Make sure you load that data and all the libraries you need in the app.R file you create. Below, you will post a link to the app that you publish on shinyapps.io. You will create an app to compare states’ cumulative number of COVID cases over time. The x-axis will be number of days since 20+ cases and the y-axis will be cumulative cases on the log scale (scale_y_log10()). We use number of days since 20+ cases on the x-axis so we can make better comparisons of the curve trajectories. You will have an input box where the user can choose which states to compare (selectInput()) and have a submit button to click once the user has chosen all states they’re interested in comparing. The graph should display a different line for each state, with labels either on the graph or in a legend. Color can be used if needed.

GitHub link

9. Below, provide a link to your GitHub page with this set of Weekly Exercises. Specifically, if the name of the file is 05_exercises.Rmd, provide a link to the 05_exercises.md file, which is the one that will be most readable on GitHub. If that file isn’t very readable, then provide a link to your main GitHub page.

DID YOU REMEMBER TO UNCOMMENT THE OPTIONS AT THE TOP?